Selecting moving objects on a system

ABSTRACT

A method of selecting a GUI object ( 118 ) in a display space ( 402 ) using a user controlled pointing device ( 104 ) is disclosed. The method comprises the steps of establishing an initial range mapping between the pointing device and the display space, and defining an initial Selected Display Region (SDR) in the display space dependent on the initial range mapping. Then the method involves receiving ( 304 ), from the pointing device, a set of coordinates in the initial SDR, updating ( 306 ) the range mapping dependent upon the received set of coordinates, and redefining the SDR dependent upon the updated range mapping. The method then determines ( 308 ) the presence of any GUI object within the redefined SDR, and if a GUI object is present in the redefined SDR, selecting ( 310 ) the GUI object.

TECHNICAL FIELD OF THE INVENTION

[0001] The present invention relates generally to selecting dynamicallymoving objects represented on a display screen and, in particular, toperforming this selection using a remote control device. The presentinvention relates to a method and apparatus for selecting a dynamicallymoving object represented on a display screen. The invention alsorelates to a computer program product including a computer readablemedium having recorded thereon a computer program for selecting adynamically moving object represented on a display screen.

BACKGROUND ART

[0002] Display devices such as televisions (TVs) often display bothstatic and dynamically moving graphical objects, as do display devicessuch as cathode ray tubes (CRTs) and Liquid Crystal Displays (LCDs) usedwith Personal Computers (PCs). These moving objects will be referred toin this description as Graphical User Interface (GUI) objects. The GUIobjects of interest in this description are graphical elements such asicons or animated graphics that represent certain information to theuser, and whose movements or positions on the display often changecontinuously in various directions or at various rates.

[0003] Common pointing devices used in PC systems include the mouse,trackball and joystick, and these are used to position a cursor at adesired position on the PC display screen. For example, the cursor maybe positioned in such a manner as to define an insertion point at whichtext is to be incorporated into a document being displayed by a wordprocessing application running on the PC. The pointing devices, andparticularly the joystick, can also be used for dynamic applicationssuch as PC based action games in which, for example, a representation ofan aeroplane can thereby be guided about the display screen.

[0004] The aforementioned pointing devices typically have a range ofoperation that enables the cursor to be moved about the entire expanseof the display screen, and a constant relationship is typicallymaintained between an operating position of the pointing device and thecorresponding location of the cursor on the display screen. In highlydynamic video applications, this arrangement, however, requires a highdegree of sensory-motor coordination from the typical user, who istherefore unable to easily track a desired dynamically moving object onthe screen.

SUMMARY OF THE INVENTION

[0005] According to one aspect of the invention, there is provided amethod of selecting a GUI object in a display space using a usercontrolled pointing device, the method comprising the steps of:

[0006] (i) establishing an initial range mapping between the pointingdevice and the display space; (ii) defining an initial Selected DisplayRegion (SDR) in the display space dependent on the initial rangemapping; (iii) receiving, from the pointing device, a set of coordinatesin the initial SDR; (iv) updating the range mapping dependent upon thereceived set of coordinates; (v) redefining the SDR dependent upon theupdated range mapping; (vi) determining a presence of any GUI objectwithin the redefined SDR; (vii) if one GUI object is present in theredefined SDR, selecting the GUI object; and

[0007] (viii) if more than one GUI object is present in the redefinedSDR, repeating steps (iii)-(viii).

[0008] According to another aspect of the invention, there is provided amethod of selecting a GUI object in a display space using a pointingdevice, the method comprising the steps of:

[0009] (a) receiving a coordinate position from the pointing device; and

[0010] (b) if a directional signal is received from the pointing device,selecting a closest GUI object in a direction dependent upon thedirectional signal.

[0011] According to another aspect of the invention, there is providedan apparatus for selecting a GUI object in a display space using a usercontrolled pointing device, the apparatus comprising:

[0012] (i) means for establishing an initial range mapping between thepointing device and the display space;

[0013] (ii) means for defining an initial Selected Display Region (SDR)in the display space dependent on the initial range mapping;

[0014] (iii) means for receiving, from the pointing device, a set ofcoordinates in the initial SDR;

[0015] (iv) means for updating the range mapping dependent upon thereceived set of coordinates;

[0016] (v) means for redefining the SDR dependent upon the updated rangemapping;

[0017] (vi) means for determining a presence of any GUI object withinthe redefined SDR;

[0018] (vii) means for selecting, if one GUI object is present in theredefined SDR, the GUI object; and

[0019] (viii) means for repeating, if more than one GUI object ispresent in the redefined SDR, steps (iii)-(viii).

[0020] According to another aspect of the invention there is provided anapparatus for selecting a GUI object in a display space using a pointingdevice, the apparatus comprising:

[0021] (a) means for receiving a coordinate position from the pointingdevice; and

[0022] (b) means for selecting, if a directional signal is received fromthe pointing device, a closest GUI object in a direction dependent uponthe directional signal.

[0023] According to another aspect of the invention there is provided acomputer program for directing a processor to execute a procedure forselecting a GUI object in a display space using a user controlledpointing device, the program comprising:

[0024] (i) code for establishing an initial range mapping between thepointing device and the display space;

[0025] (ii) code for defining an initial Selected Display Region (SDR)in the display space dependent on the initial range mapping;

[0026] (iii) code for receiving, from the pointing device, a set ofcoordinates in the initial SDR;

[0027] (iv) code for updating the range mapping dependent upon thereceived set of coordinates;

[0028] (v) code for redefining the SDR dependent upon the updated rangemapping;

[0029] (vi) code for determining a presence of any GUI object within theredefined SDR;

[0030] (vii) code for selecting, if one GUI object is present in theredefined SDR, the GUI object; and

[0031] (viii) code for repeating, if more than one GUI object is presentin the redefined SDR, steps (iii)-(viii).

[0032] According to another aspect of the invention there is provided acomputer program for directing a processor to execute a procedure forselecting a GUI object in a display space using a pointing device, theprogram comprising:

[0033] (a) code for receiving a coordinate position from the pointingdevice; and

[0034] (b) code for selecting, if a directional signal is received fromthe pointing device, a closest GUI object in a direction dependent uponthe directional signal.

[0035] According to another aspect of the invention there is provided acomputer program product including a computer readable medium havingrecorded thereon a computer program for directing a processor to executea procedure for selecting a GUI object in a display space using a usercontrolled pointing device, the program comprising:

[0036] (i) code for establishing an initial range mapping between thepointing device and the display space;

[0037] (ii) code for defining an initial Selected Display Region (SDR)in the display space dependent on the initial range mapping;

[0038] (iii) code for receiving, from the pointing device, a set ofcoordinates in the initial SDR;

[0039] (iv) code for updating the range mapping dependent upon thereceived set of coordinates;

[0040] (v) code for redefining the SDR dependent upon the updated rangemapping;

[0041] (vi) code for determining a presence of any GUI object within theredefined SDR;

[0042] (vii) code for selecting, if one GUI object is present in theredefined SDR, the GUT object; and

[0043] (viii) code for repeating, if more than one GUI object is presentin the redefined SDR, steps (iii)-(viii).

[0044] According to another aspect of the invention there is provided acomputer program product including a computer readable medium havingrecorded thereon a computer program for directing a processor to executea procedure for selecting a GUI object in a display space using apointing device, the program comprising:

[0045] (a) code for receiving a coordinate position from the pointingdevice; and

[0046] (b) code for selecting, if a directional signal is received fromthe pointing device, a closest GUI object in a direction dependent uponthe directional signal.

[0047] Other aspects of the invention are also disclosed.

BRIEF DESCRIPTION OF THE DRAWINGS

[0048] One or more embodiments of the present invention will now bedescribed with reference to the drawings, in which:

[0049]FIG. 1 shows a block diagram of a system in which a user canselect a GUI object on a display screen;

[0050]FIG. 2 is a schematic block diagram of a general-purpose computerupon which arrangements described can be practiced;

[0051]FIG. 3A shows an illustrative the touch pad device;

[0052]FIG. 3B shows an initial Selected Display Region (SDR);

[0053]FIG. 3C shows a subsequent SDR after revised range mapping;

[0054]FIG. 3D shows an alternative range mapping arrangement;

[0055]FIG. 4 shows a process whereby a user can operate the system ofFIG. 1 for selecting GUI objects by providing positional inputs;

[0056]FIG. 5 depicts a system process flow chart of method steps bywhich the software application in FIG. 1 selects the GUI object usingpositional inputs;

[0057]FIG. 6 depicts selective magnification of the Selected DisplayRegion (SDR) on PC display screen;

[0058]FIG. 7 shows a process whereby a user can operate the system ofFIG. 1 for selecting objects by providing directional inputs;

[0059]FIG. 8 illustrates one arrangement for mathematically defining aclosest object in the specified direction; and

[0060]FIG. 9 depicts a system process flow chart of method steps bywhich the application of FIG. 1 selects the GUI object using directionalinputs.

DETAILED DESCRIPTION INCLUDING BEST MODE

[0061] Where reference is made in any one or more of the accompanyingdrawings to steps and/or features, which have the same referencenumerals, those steps and/or features have for the purposes of thisdescription the same function(s) or operation(s), unless the contraryintention appears.

[0062] As previously noted, it is often desirable for the user to havethe capability of selecting one of a possibly large number of GUIobjects that are moving dynamically on a display screen of a PC systemor a TV system that has an associated PC capability.

[0063]FIG. 1 shows a block diagram of a system 100 by which a user 102may select a GUI object 118 on a display area 402 of a display device112. The user 102 uses a remote control device 104 in order to performthe aforementioned selection. The exemplary GUI object 118 has adirection and a velocity depicted by a vector 120. GUI objects typicallyrepresent information specific to an application 116 that is being runon a PC module 201. The exemplary remote control device 104 has aninfra-red (IR) transmitter which converts user inputs 106 tocorresponding commands 110 directed to the PC module 201. The exemplaryremote control device 104 described in this specification is a touch paddevice 410 equipped with a touch pad region 416 as well as othercontrols (see FIG. 3A). Other devices, however, such as joy-sticks,multi-way rocker switches, or other controls that provide positionaland/or directional inputs may alternately be used.

[0064] When selected the GUI object 118 can be highlighted on thedisplay area 402 in order to provide feedback to the user 102 thatselection has been performed. The software application 116 generates thedynamic GUI objects that are displayed on the display device 112,interprets the commands 110, and sends corresponding graphical data 114to the display device 112, thereby changing the display of the GUIobjects accordingly. The graphical data 114 can highlight a GUI objectthat has been thus selected by the user 102, thereby providing visualfeedback 108 from the display device 112 to the user 102, and providingthe user 102 with confirmation that the desired GUI object has beensuccessfully selected.

[0065]FIG. 2 illustrates how the method of selecting the GUI object ispreferably practiced using a general-purpose computer system 200. Inthis arrangement, processes described in relation to FIGS. 4, 5, 7, 9may be implemented as software, such as the application program 116executing within the computer system 200. In particular, the steps ofmethod of selecting a GUI object are effected by instructions in thesoftware that are carried out by the computer. The instructions may beformed as one or more code modules, each for performing one or moreparticular tasks. The software may also be divided into two separateparts, in which a first part performs the selection of GUT objectmethods, and a second part manages a user interface between the firstpart and the user. The software may be stored in a computer readablemedium, including the storage devices described below, for example. Thesoftware is loaded into the computer from the computer readable medium,and then executed by the computer. A computer readable medium havingsuch software or computer program recorded on it is a computer programproduct. The use of the computer program product in the computerpreferably effects an advantageous apparatus for selecting a desired GUIobject.

[0066] The computer system 200 comprises the computer module 201, inputdevices such as a keyboard 202, mouse 203, and the touch pad device 410,output devices including a printer 215 and the display device 112 havingthe display area 402. A Modulator-Demodulator (Modem) transceiver device216 is used by the computer module 201 for communicating to and from acommunications network 220. The modem is, for example, connectable tothe network 220 via a telephone line 221 or other functional medium. Themodem 216 can be used to obtain access to the Internet, and othernetwork systems, such as a Local Area Network (LAN) or a Wide AreaNetwork (WAN).

[0067] The computer module 201 typically includes at least one processorunit 205, a memory unit 206, for example formed from semiconductorrandom access memory (RAM) and read only memory (ROM). The module 201also includes input/output (I/O) interfaces, including a video interface207, and an I/O interface 213 for the keyboard 202, the mouse 203, thetouch pad device 410 by means of a wireless connection 224, andoptionally a joystick (not illustrated). The module 201 also has aninterface 208 for the modem 216. A storage device 209 is provided andtypically includes a hard disk drive 210 and a floppy disk drive 211. Amagnetic tape drive (not illustrated) may also be used. A CD-ROM drive212 is typically provided as a non-volatile source of data Thecomponents 205 to 213 of the computer module 201, typically communicatevia an interconnected bus 204 and in a manner which results in aconventional mode of operation of the computer system 200 known to thosein the relevant art. Examples of computers on which the describedarrangements can be practised include IBM-PC's and compatibles, SunSparcstations or alike computer systems evolved therefrom.

[0068] Typically, the application program 116 is resident on the harddisk drive 210 and read and controlled in its execution by the processor205. Intermediate storage of the program and any data fetched from thenetwork 220 may be accomplished using the semiconductor memory 206,possibly in concert with the hard disk drive 210. In some instances, theapplication program may be supplied to the user encoded on a CD-ROM orfloppy disk and read via the corresponding drive 212 or 211, oralternatively may be read by the user from the network 220 via the modemdevice 216. Still further, the software can also be loaded into thecomputer system 200 from other computer readable media The term“computer readable medium” as used herein refers to any storage ortransmission medium that participates in providing instructions and/ordata to the computer system 200 for execution and/or processing.Examples of storage media include floppy disks, magnetic tape, CD-ROM, ahard disk drive, a ROM or integrated circuit, a magneto-optical disk, ora computer readable card such as a PCMCIA card and the like, whether ornot such devices are internal or external of the computer module 201.Examples of transmission media include radio or infra-red transmissionchannels as well as a network connection to another computer ornetworked device, and the Internet or Intranets including emailtransmissions and information recorded on websites and the like.

[0069] The method of selecting a GUI object may alternatively beimplemented in dedicated hardware such as one or more integratedcircuits performing the functions or sub functions of selecting a GUIobject. Such dedicated hardware may include graphic processors, digitalsignal processors, or one or more microprocessors and associatedmemories.

[0070] Two variations of the method of selecting a GUI object aredescribed, each requiring the user 102 to use the touch pad device 410in a slightly different manner, and for which the display device 112operates in correspondingly different feedback modes. Preparatory todescribing the aforementioned selection methods, it is convenient todescribe the exemplary touch pad device 410 that is used as the remotecontrol device 104.

[0071]FIG. 3A illustrates the touch pad device 410 that is described inthis specification. The touch-pad device comprises an outer casing 432and a number of controls. It can also include a small display (notshown) that mirrors, typically at a lower resolution, the display on thedisplay area 402 of the display device 112. In general, controls on theremote control device 104 are used for making GUI object selection, andalso for other purposes that are specific to the exemplary application116. In the present example, a touch pad region 416 serves as thecontrol for providing positional or directional information that is usedto select the desired GUI object. A MULTI-SELECT toggle button 414serves to enable or disable multiple selections of GUI objects, and anEXIT button 430 allows the user to exit an operation in a currentcontext.

[0072] The touch pad device 410 allows the user 102 to specifypositional information in the two-dimensional display space 402, interms of a pair of coordinates, by touching a position on the touch padregion 416 with a finger or a stylus. The touch pad region 416 alsoallows the user 102 to specify directional information in thetwo-dimensional display space 402. Directional information is specifiedin terms of an initial finger placement coordinate on the touch padregion 416, and an angle defined by holding the user holding theirfinger down and dragging their finger in a desired direction along thesurface of the touch pad region 416. An imaginary line (see 908 in FIG.8) formed between the initial finger placement position and the fingerposition after the dragging action is completed is used to derive anangle relative to an axis.

[0073] The MULTI_SELECT 414 button is a toggle button which has an ON orOFF state. When OFF, only one GUI object can be selected at any onetime. When a GUI object is selected, any previously selected GUI objectsare de-selected. When the button 414 is ON, a new selection can be madewithout de-selecting any previous GUI object selections. The EXIT buttonis used to exit an operation in the current context so that it can becancelled or retried. Thus, for example, if the user no longer wishes toselect a GUI object, or if the desired GUI object is outside a SelectedDisplay Region (SDR), the user can use the EXIT button to revert to thestate just prior to inputting the last position. This allows the user tocancel or retry the last operation. If the user wishes to make multipleselections of GUI objects, the MULTI-SELECT button has to be toggled ONbefore further selections are made. Otherwise, the next selection willde-select the current selection. While selecting multiple GUI objects,selecting an already selected GUI object will de-select it.

[0074] In a first arrangement, the user provides what are referred to as“positional inputs”. This allows the user 102 to progressively performselection, while maintaining the complete dynamic range of the touch paddevice 410, in a smaller and smaller display region, referred to as aSelected Display Region (SDR), on the display area 402 (see FIG. 3B) ofthe display device 112. This process continues until the SDR containsonly the desired GUI object.

[0075]FIG. 3B shows a first instance of the display device 112 and thedisplay area 402, in regard to which a technique called “range mapping”between the display area 402 of the display device 112 and the touch padregion 416 of the touch pad device 410 will now be described. In thecase when positional inputs are used to select a GUI object, theselection process commences by performing an initial range mapping ofthe touch pad region 416 of the touch pad 410 (see FIG. 3A) to theentire display area 402. This initial range mapping defines an initialSDR that encompasses the entire display area 402. The initial SDR isthus depicted by an elliptical boundary 404. The touch pad region 416 isdivided into nine touch regions exemplified by touch regions 412 and420, and in the initial range mapping, these nine touch regions map tonine corresponding display regions exemplified by display regions 400and 406 on the display area 402. The term range mapping denotes therelationship between the display area 402, a current SDR, and the touchpad region 416, and denotes how a touch on the touch pad region 416 isinterpreted by the application 116 as corresponding to a particulardisplay region in the current SDR of the display area 402. In accordancewith the initial range mapping, a user touch input to the top left handtouch region 412 of the touch pad region 416 is interpreted ascorresponding to the top left hand display region 400 of the SDR 404.Similarly, an input to the bottom right hand touch region 420 of thetouch pad region 416 is interpreted as corresponding to the bottom righthand display region 406 of the SDR 404.

[0076] If the user notices a desired GUI object, say in the displayregion 400 of the initial SDR 404, the user 102 touches a point on thetouch region 412 of the touch pad region 416 that corresponds to anapproximate position of the desired GUI object in the display region 400in the current SDR 404. The touch pad device 410 then transmits acorresponding command 110 that represents the location of the touchregion 412 that has been pressed by the user, to the application 116. Inone arrangement, the application 116 determines whether there is asingle GUI object only in the display region 400. If this is the case,the application 116 selects that single GUI object in the display region400, and that GUI object is highlighted in the display to show the userthat the GUI object has been selected. If, however, there are multipleGUI objects in the display region 400, then none of these GUI items arehighlighted, and the application 116 revises the range mapping andre-maps the touch pad region 416 to just the specified display region400, thereby specifying a new SDR 404′.

[0077]FIG. 3C shows a second instance of the display device 112 and thedisplay area 402, after the aforementioned revised range mapping. Asnoted, the new SDR is now defined by the elliptical boundary 404′, andthe rectangular boundary of the display region 400 is highlighted asdepicted in FIG. 3C by a bold border. The touch pad region 416 of thetouch pad 410 is re-mapped by the application 116 to only thehighlighted new SDR 404′. The practical effect of the aforementionedhighlighting is to provide the user with immediate visual feedback as tothe region of the display to which the user has pointed, and thisprovides a new frame of reference to the user for inputting the nextposition on the touch pad 410. The revised range mapping results in thefact that an input to the touch region 412 of the touch pad region 416is now interpreted as corresponding to a display region 422 of the SDR404′. Similarly, an input to the touch region 420 of the touch padregion 416 is interpreted as corresponding to a display region 428 ofthe new SDR 404′.

[0078] If the desired GUI object is seen by the user to be, say, in thedisplay region 422 of the new SDR 404′, the user 102 touches a point onthe touch region 412 of the touch pad region 416 that corresponds to anapproximate position of the desired GUI object in the display region 422in the new SDR 404′. The touch pad device 410 the transmits acorresponding command 110 that represents the location of the touchregion 412 that has been pressed by the user, to the application 116. Inthe described arrangement, the application 116 determines whether thereis a single GUI object only in the display region 422. If this is thecase, the application 116 selects that single GUI object in the displayregion 422, and that GUI object is highlighted in the display to showthe user that the GUI object has been selected. If, however, there aremultiple GUI objects in the display region 422, then the application 116revises the range mapping and re-maps the touch pad region 416 to justthe specified display region 422, thereby specifying yet a new SDR 404″.

[0079] In the described arrangement, if there is only one GUI object ina particular display region that the user has pointed to, then that GUIobject is selected and highlighted. The highlighting of the GUI objectindicates to the user that he has actually selected the highlighted GUIobject. If, however, there are multiple GUI objects in the displayregion that has been pointed to, then no GUI object is highlighted. Arevised range mapping is performed, and a new SDR is thereby defined andis also highlighted. The user can again touch a point on the touch padregion 416 that corresponds, in the estimation of the user 102, to thecoordinate position of the GUI object in the newly highlighted SDR. Theuser continues with the above process until the desired GUI object isselected.

[0080] The described arrangement highlights the SDR and the desired GUIobject in the display area 402 in order to provide visual feedback cuesto the user. These cues help the user track and select the GUI object.Other feedback indications can also be used. For example, indicativetext messages can be provided, either on the display area 402, or onanother auxiliary display (not shown).

[0081] In general, the range-mapping technique enables the touch padregion 416 of the touch pad device 410 to be mapped to an SDR comprisingthe entire display area 402, as well as to new SDRs encompassing anysub-section of the display region 402. Successive range mappingoperations thus provide the user of the touch pad 410 with a convenientand effective method for rapidly converging on a desired GUI objectdisplayed in the display area 402, while maintaining the full dynamicrange of the touch pad region 416 over successively smaller SDRs.

[0082] It is apparent that although the arrangement in FIG. 3 dealsspecifically with a two-dimensional (2-D) display area 402, and acorresponding 2-D pointing device in the form of the touch pad 410, thisis merely an example. Thus the aforementioned range-mapping concept canbe applied to an N-dimensional display space and a correspondingN-dimensional pointing device.

[0083] A three-dimensional arrangement could be implemented, forexample, using a “virtual-reality” display arrangement together with ajoy stick having auxiliary controls to enable three-dimensionalpointing.

[0084]FIG. 3D shows an alternative range mapping arrangement, whereinrange mapping is performed on the basis of a user selected coordinateposition of the desired GUI object, and the actual coordinate positionof that GUI object which is closest to the user selected coordinateposition, thus not relying on the 9-segment touch region as the basisfor defining new SDRs. The display device 112 and the display area 402show three displayed GUI objects 1000, 1006 and 1008. The touch padregion 416 is initially mapped to the entire display area 402, whichthus forms the initial SDR. The user selects a coordinate position 1004by touching a particular (corresponding) position of the touch padregion 416. The touch pad device 410 transmits a command 110 thatrepresents the location of the touch region 412 that has been pressed bythe user, to the application 116. The application 116 then determineswhich is the closest GUI object to the user selected coordinateposition, which in this case is the GUI object 1008 which is closer tothe user selected coordinate position 1004 than other displayed GUIobjects 1000 and 1006. The application 116 then defines a new SDR 1010having the aforementioned closest GUI object 1008 at the centre of thenew SDR, and the user selected coordinate 1004 near a boundary thereof.The application 116 also revises the range mapping and re-maps the touchpad region 416 to the new SDR 1010.

[0085]FIG. 4 shows a process 300 whereby the user 102 can operate thesystem of FIG. 1 for selecting the desired GUI object by providingpositional inputs. The process commences at a step 302 at which stagethe user 102 is ready to select the desired GUI object represented onthe display area 402 of the display device 112. Thereafter, at a step304 the user inputs an estimated coordinate position of the desired GUIobject using the touch pad region 416 of the touch pad device 410.Thereafter, a testing step 314 determines whether there is a GUI objectin the SDR. If there is no such object, then the process 300 is directedin accordance with a “yes” arrow back to the step 304. If, on the otherhand, a GUI object is to be found in the SDR, then the process 300 isdirected in accordance with a “no” arrow to a step 306, in which theapplication 116 highlights the perimeter of the new SDR resulting fromthe user input, using the range mapping process. The step 306 alsoremaps the touch pad region 416 of the touch pad 410 to just thehighlighted new SDR. The remapping ensures that the entire dynamic rangeof the touch pad device 410 is applied to the new SDR, thereby providingthe user 102 with a finer granularity of control within the new SDR.

[0086] In a following testing step 308 the application 116 determineswhether there is only one GUI object in the SDR that has now beenhighlighted on the display area 402. If there is only one displayed GUIobject in the SDR, then the process is directed in accordance with a“yes” arrow to a step 310 in which the application 116 selects andhighlights that GUI object. The highlighting operation provides the user102 with visual feedback 108 that he has, indeed, selected the desiredGUI object. The process 300 thereafter terminates at a step 312.

[0087] Returning to the testing step 308, if there are GUI objects otherthan the desired GUI object in the SDR, then the process 300 is directedin accordance with a “no” arrow to the step 304 where the user providesanother positional input to the touch pad device 410.

[0088]FIG. 5 shows depicts a system process flow chart of method stepsby which the application 116 of FIG. 1 selects the desired GUI objectusing the positional inputs. The process 700 commences with a step 702which establishes an (initial) range mapping between the display area402 of the display device 112 and the selectable range of the touch paddevice 410. The process 700 is then directed to a step 704 that receivesestimated coordinate information, referred to the display area 402, forthe desired GUI object. Thereafter, a step 706 determines a new rangemapping between the display area 402 and the selectable range of thetouch pad device 410. Thereafter, the step 708 determines a new SDRbased upon the new range mapping, and the step 708 also highlights thenew SDR to provide a visual feedback indication. Thereafter, a testingstep 722 determines if a GUI object is to be found in the SDR. If no GUIobject is to be found, then the process 700 is directed in accordancewith a “yes” arrow back to the step 704. If, on the other hand, a GUIobject is to be found in the SDR, then the process 700 is directed inaccordance with a “no” arrow to a subsequent step 710 which remaps thetouch pad region 416 of the touch pad 410 to just the aforementionedhighlighted SDR.

[0089] Thereafter, the testing step 712 ascertains whether objects otherthan the desired GUI object are present in the highlighted current SDR.If no other objects are detected, then the process 700 is directed inaccordance with “yes” arrow to a step 714 which highlights therepresentation of the GUI object, at which point the process 700terminates. If, on the other hand, other objects are detected in thestep 712, then the process 700 is directed in accordance with a “no”arrow to the step 704 which receives a (new) set of estimatedcoordinates for the desired GUI object.

[0090] The steps 704-712 are repeated until a single object is presentin the current SDR, at which point the process 700 is directed inaccordance with the “yes” arrow to the step 714 which highlights thedesired GUI object and terminates the process.

[0091] The step 702 can be implemented, in one arrangement indicated byreference numerals contained in a dashed box 718 connected by a dashedline 720 to the step 702 in question, by the processor 205, inconjunction with the touch pad device 410 and the display device 112, ascan the step 710. Similarly, the step 704 can be implemented by thetouch pad device 410, in conjunction with the processor 205. Similarly,the steps 706 and 712 can be implemented by the processor 205.Similarly, the steps 708 and 714 can be implemented by the processor205, in conjunction with the video interface 207 and the display device112.

[0092] As the user 102 directs her attention to successively smallerSDRs 404, 404′, 404″, . . . , the user must direct her attention tosuccessively smaller segments of the display area 402 and visualresolution decreases. The SDR can however be enlarged on the display toshow each SDR in magnified form, preferably using the entire displayarea 402. This displays GUI objects in greater detail.

[0093]FIG. 6 shows this aforementioned arrangement whereby as theattention of the user 102 is directed from a current SDR 502 to a newSDR 500, the new SDR 500 is magnified and mapped to the entire displayarea 402 as depicted by dashed lines 504, 508, 506 and 510. Thisprovides the user 102 with a magnified view of contents of the new SDR500 as shown. In order to provide the user with a frame of reference asto where the magnified new SDR 500 lies in regard to the overall displayarea 402, a reduced representation of both the overall display area 402and the new SDR 500 (called a “picture-in-picture view”) can be shown,at 516 and 514. This picture-in-picture view can be intelligently placedat one of the corners of the display area 402 to avoiding obstructingany of the GUI objects.

[0094] Having described selection of GUI objects using positionalinformation, we turn to a second arrangement in which the user 102 canselect a GUI object using Directional Inputs. In this arrangement, thetouch pad region 416 of the touch pad device 410 is mapped to the entiredisplay area 402 (see FIG. 3B). When the user observes the display 402and notices a desired GUI object, he touches an initial position on thetouch pad region 416 that corresponds to the perceived position of thedesired GUI object on the display 402, and holds his finger down on thetouch pad region 416 at the initially selected position. The touch paddevice 410 transmits the coordinates of the touched position to theapplication 116. The application determines which is the nearest GUIobject to the specified position and marks that GUI object in the memory206 as a “targeted” GUI object. This currently targeted GUI object ishighlighted in the display 402 to provide the user 102 with visualfeedback of which GUI object has been targeted. If the targeted GUIobject is not the desired GUI object, the user uses the touch pad device410 to indicate a direction to the desired GUI object by dragging hisfinger in the direction of that desired GUI object relative to thecurrently targeted and highlighted GUI object. The touch pad device 410transmits the specified direction to the application 116 that thenselects and highlights the GUI object that is nearest to the currentlytargeted GUI object in the specified direction. If that newlyhighlighted GUI object is the desired one, the user then lifts hisfinger off the touch pad to finalise the selection thereof.

[0095]FIG. 7 shows a process 600 for this arrangement whereby the user102 can select a GUI object using directional input information. Theprocess 600 commences at a step 602, at which juncture the user 102 isready to select the desired GUI object. Thereafter, a step 604 receivesan (initial) set of estimated coordinates for the desired GUI objectfrom the user 102 who is using the touch pad device 410. Thereafter, ata step 606 the GUI object nearest to the aforementioned estimatedcoordinate position is identified, and in a following step 608, theidentified GUI object is targeted and highlighted, thereby indicating tothe user 102 which object has been designated.

[0096] In a following step 614, the user determines whether the targeted(highlighted) GUI object is, indeed, the desired object. If this is thecase, then the process 600 is s directed in accordance with a “yes”arrow to a step 616. At that point, the user 102 releases control of thetouch pad 410, finalising the selection of the desired GUI object, andthe process 600 then terminates at a step 622.

[0097] Returning to the testing step 614, if the highlighted GUI objectis seen by the user 102 not to be the desired GUI object, then theprocess 600 is directed in accordance with a “no” arrow to a step 612.The user has up to this point kept his or her finger on the touch padregion 416. The user maintains his finger in contact with the touch padregion 416 and “drags” his finger in the direction of the desired GUIobject. In a subsequent step 610 the application 116 identified a GUIobject that is “nearest” to the currently identified GUI object in thespecified direction. This is described in more detail in regard to FIG.8. The process 600 is then directed back to the step 608 whichhighlights the newly identified GUI object.

[0098] The process 600 cycles through the steps 608, 614, 612 and 610,effectively pursuing the desired GUI object under the guidance ofdirectional inputs provided by the user.

[0099]FIG. 8 shows one arrangement 900 by which directional inputsprovided by the user 102 can be used in order to select a GUI objectwhich is nearest to the currently identified GUI object, in thedirection specified by the user. In FIG. 8, an object 920 is the desiredGUI object, however the user 102 has, by touching the touch pad region416, indicated a location on the display area 402 corresponding to thatof a cursor 904 as shown. Having regard to the object 920, and to otherobjects 914 and 902, the cursor 904 is closest to the object 902 asdepicted by an arrow 906, and accordingly the object 902 is highlightedby the application 116. The user 102 receives the visual feedback 108 byvirtue of this highlighting, and drags his finger along the touch padregion 416 thereby establishing a direction vector 908 originating fromthe cursor 904. While the user is dragging his or her finger along thetouch pad, or, for that matter while the user's finger is stationaryupon the touch pad, noise can be generated by slight unintentionalmovements of the user's finger. Such noise can potentially have theundersirable effect of causing a different GUI object to be highlighted,rather than the desired GUI object. Noise filtering can, however, easilybe applied when the present arrangement is implemented, therebyovercoming this problem. Thus, for example, any dragging movement whosemagnitude falls below a certain threshold can be ignored. The directionvector 908 points along a dashed line 916 that is shown for convenience.The object 914 is closer to the cursor 904 than is the object 920, asshown by arrows 912 and 918 respectively. However, the object 914 fallsoutside the triangle 910 that establishes a maximum angular deviationabout the line 916 into which GUI objects must fall in order to beselected. Accordingly, the GUI object 920 is deemed to be the closestGUI object to the cursor 904 because the GUI object 920 falls within themaximum angular deviation established by the triangle 910. Accordingly,the GUI object 920 is highlighted, and this information is fed backvisually to the user 102.

[0100]FIG. 9 shows a system process 800 of method steps by which theapplication 116 running on the PC system 200 selects a GUI object on thedisplay device 112 using directional input information from the user102. The process 102 commences at a step 802 that receives an estimatedcoordinate position of the desired GUI object. Thereafter, a step 804highlights a GUI object on the display device 112 that is closest to theaforementioned estimated coordinate position. A subsequent testing step806 establishes whether the highlighted GUI object is, indeed, thedesired GUI object. If this is the case, then the process 800 isdirected in accordance with a “yes” arrow to a step 814 at which pointthe process 800 terminates. If, on the other hand, the highlightedobject is not the desired object, then the process 800 is directed inaccordance with a “no” arrow to a step 808 which receives a directionalinput that points towards the desired GUI object. The process 800 isthen directed to a step 810 which highlights the “closest” GUI object,having regard to the aforementioned directional input. The process 800is then directed in accordance with an arrow 812 back to the testingstep 806.

[0101] The step 802 can be implemented, in one arrangement indicated byreference numerals contained in a dashed box 816 connected by a dashedline 818 to the step 802 in question, by the processor 205, inconjunction with the touch pad device 410, as can the step 808.Similarly, the step 804 can be implemented by the processor 205, inconjunction with the video interface 207 and the display device 112.Similarly, the steps 806 and 814 can be implemented by the processor205. Similarly, the step 808 can be implemented by the touch pad device410, in conjunction with the processor 205.

[0102] Industrial Applicability

[0103] It is apparent from the above that the arrangements described areapplicable to the image processing industry.

[0104] The foregoing describes only some embodiments of the presentinvention, and modifications and/or changes can be made thereto withoutdeparting from the scope and spirit of the invention, the embodimentsbeing illustrative and not restrictive.

[0105] There may be variations of the method depending on the type ofcontrol devices available and the user's preference or level ofcoordination. Thus, although the use of a touch pad device 410 isconsidered in the detailed description, other controls which providespositional and/or direction input can also be adopted. For example, ajoystick or a multi-way rocker switch can be used instead for thepositional input arrangement. For a joystick, one of the nine regionscan be selected by moving the joystick to one of the nine positions andthen pressing a joystick button. For a multi-way rocker switch, one ofthe nine regions can be selected by pressing the switch in one of theeight positions or the centre position.

[0106] Furthermore, if the remote control device is equipped with amini-touch screen, a low-resolution image can be displayed on it toprovide the user with better precision when inputting a position.

[0107] Furthermore, while the user is making a selection, the GUIobjects are typically still moving in real-time, driven by theapplication 116. In order for the user to complete a selection, all theGUI objects in the SDR can be temporarily frozen until a GUI objectselection is made.

The claims defining the invention are as follows:
 1. A method ofselecting a GUI object in a display space using a user controlledpointing device, the method comprising the steps of: (i) establishing aninitial range mapping between the pointing device and the display space;(ii) defining an initial Selected Display Region (SDR) in the displayspace dependent on the initial range mapping; (iii) receiving, from thepointing device, a set of coordinates in the initial SDR; (iv) updatingthe range mapping dependent upon the received set of coordinates; (v)redefining the SDR dependent upon the updated range mapping; (vi)determining a presence of any GUI object within the redefined SDR; (vii)if one GUI object is present in the redefined SDR, selecting the GUIobject; and (viii) if more than one GUI object is present in theredefined SDR, repeating steps (iii)-(viii).
 2. A method according toclaim 1, wherein: step (vii) has an additional sub-step of providing afeedback signal to thereby identify the selected GUI object; and themethod comprises a further step of: (ix) if no GUI objects are presentin the redefined SDR, repeating steps (i) to (ix).
 3. A method accordingto claim 1, wherein the GUI object in the step (vii) is the only GUIobject in the redefined SDR.
 4. A method according to claim 1, whereinstep (v) has an additional sub-step of providing a feedback signal tothereby identify the redefined SDR.
 5. A method according to claim 1,wherein step (iii) comprises an initial sub-step of: freezing motion ofGUI objects in the display space until a coordinate position is receivedfrom the pointing device.
 6. A method according to claim 1, wherein theSDR is displayed in a magnified format in the display space.
 7. A methodaccording to claim 1, wherein the display space is N-dimensional.
 8. Amethod according to claim 7, wherein: N=2; and said pointing device is atouch pads.
 9. A method of selecting a GUI object in a display spaceusing a pointing device, the method comprising the steps of: (a)receiving a coordinate position from the pointing device; and (b) if adirectional signal is received from the pointing device, selecting aclosest GUI object in a direction dependent upon the directional signal.10. A method according to claim 8, wherein the display space isN-dimensional.
 11. A method according to claim 10, wherein: N=2; andsaid pointing device is a touch pad.
 12. A method according to claim 9,wherein at least one of step (a) and step (b) comprises an initialsub-step of freezing motion of GUI objects in the display space until arespective coordinate position and directional signal is received fromthe pointing device.
 13. A method according to claim 9, wherein when anew GUI object is selected, a previously selected GUI object isde-selected.
 14. A method according to claim 9, wherein when a new GUIobject is selected, a previously selected GUI object is not de-selected.15. An apparatus for selecting a GUI object in a display space using auser controlled pointing device, the apparatus comprising: (i) means forestablishing an initial range mapping between the pointing device andthe display space; (ii) means for defining an initial Selected DisplayRegion (SDR) in the display space dependent on the initial rangemapping; (iii) means for receiving, from the pointing device, a set ofcoordinates in the initial SDR; (iv) means for updating the rangemapping dependent upon the received set of coordinates; (v) means forredefining the SDR dependent upon the updated range mapping; (vi) meansfor determining a presence of any GUI object within the redefined SDR;(vii) means for selecting, if one GUI object is present in the redefinedSDR, the GUI object; and (viii) means for repeating, if more than oneGUI object is present in the redefined SDR, steps (iii)-(viii).
 16. Anapparatus according to claim 15, wherein the GUI object in the step(vii) is the only GUI object in the redefined SDR.
 17. An apparatus forselecting a GUI object in a display space using a pointing device, theapparatus comprising: (a) means for receiving a coordinate position fromthe pointing device; and (b) means for selecting, if a directionalsignal is received from the pointing device, a closest GUI object in adirection dependent upon the directional signal.
 18. A computer programfor directing a processor to execute a procedure for selecting a GUIobject in a display space using a user controlled pointing device, theprogram comprising: (i) code for establishing an initial range mappingbetween the pointing device and the display space; (ii) code fordefining an initial Selected Display Region (SDR) in the display spacedependent on the initial range mapping; (iii) code for receiving, fromthe pointing device, a set of coordinates in the initial SDR; (iv) codefor updating the range mapping dependent upon the received set ofcoordinates; (v) code for redefining the SDR dependent upon the updatedrange mapping; (vi) code for determining a presence of any GUI objectwithin the redefined SDR; (vii) code for selecting, if one GUI object ispresent in the redefined SDR, the GUI object; and (viii) code forrepeating, if more than one GUI object is present in the redefined SDR,steps (iii)-(viii).
 19. A computer program according to claim 18,wherein the GUI object in the step (vii) is the only GUI object in theredefined SDR.
 20. A computer program for directing a processor toexecute a procedure for selecting a GUI object in a display space usinga pointing device, the program comprising: (a) code for receiving acoordinate position from the pointing device; and (b) code forselecting, if a directional signal is received from the pointing device,a closest GUI object in a direction dependent upon the directionalsignal.
 21. A computer program product including a computer readablemedium having recorded thereon a computer program for directing aprocessor to execute a procedure for selecting a GUI object in a displayspace using a user controlled pointing device, the program comprising:(i) code for establishing an initial range mapping between the pointingdevice and the display space; (ii) code for defining an initial SelectedDisplay Region (SDR) in the display space dependent on the initial rangemapping; (iii) code for receiving, from the pointing device, a set ofcoordinates in the initial SDR; (iv) code for updating the range mappingdependent upon the received set of coordinates; (v) code for redefiningthe SDR dependent upon the updated range mapping, (vi) code fordetermining a presence of any GUI object within the redefined SDR; (vii)code for selecting, if one GUI object is present in the redefined SDR,the GUI object; and (viii) code for repeating, if more than one GUIobject is present in the redefined SDR, steps (iii)-(viii).
 22. Acomputer program product according to claim 21, wherein the GUI objectin the step (vii) is the only GUI object in the redefined SDR.
 23. Acomputer program product including a computer readable medium havingrecorded thereon a computer program for directing a processor to executea procedure for selecting a GUI object in a display space using apointing device, the program comprising: (a) code for receiving acoordinate position from the pointing device; and (b) code forselecting, if a directional signal is received from the pointing device,a closest GUI object in a direction dependent upon the directionalsignal.